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ABSTRACT 

The  need  for  an  auxiliary  performance  measure  for  the 
design  of  constrained  optimal  controls  for  linear  regula- 
tors is  shown.   Several  auxiliary  performance  measures  are 
compared,  and  the  maximum  of  the  absolute  degradation  over 
the  admissible  initial  states  is  selected  as  the  auxiliary 
performance  measure.   Computational  algorithms  which  make 
extensive  use  of  existing  library  subprograms  are  developed 
for  the  design  of  constrained  optimal  controls  in  those 
cases  where  the  control  vector  is  specified  as  constant  or 
piecewise-constant  linear  feedback  of  the  output  vector. 
Numerical  examples  including  a  third-order  system  and  a 
time-varying  system  are  given  to  illustrate  the  applica- 
tions of  the  proposed  algorithms. 
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I.   INTRODUCTION 

The  linear  regulator  is  one  of  the  most  extensively 
studied  and  well-known  problems  of  optimal  control  theory 
[1,2,3,4],   The  importance  of  linear  regulators  comes  from 
the  fact  that  many  practical  control  problems  can  be  formu- 
lated in  the  form  of  a  linear  regulator.   In  addition,  the 
feedback  form  of  the  optimal  solution  to  the  problem  is  a 
very  desirable  feature.   The  practical  application  of  the 
optimal  solution  to  the  linear  regulator  problem  suffers 
from  two  serious  drawbacks.   One  is  the  time-varying  nature 
of  the  feedback  gains,  and  the  other  is  the  need  for  exact 
physical  measurements  of  the  value  of  the  state  vector  at 
every  instant  of  time.   In  general,  not  the  state  vector  of 
the  system,  but  the  output  vector  of  the  system  is  available 
for  measurements.   It  is  possible  to  reconstruct  the  state 
vector  [5],  or  to  obtain  an  approximation  to  the  state  vec- 
tor that  is  best  in  some  sense  when  noise  is  present  [6]  ; 
however,  in  many  cases,  these  schemes  to  obtain  the  state 
vector  are  not  justified  due  to  economic  considerations. 

Since  the  mathematical  formulation  and  solution  of  the 
optimal  linear  regulator  problem  by  Kalman  [2],  the  design 
of  suboptimal  controls,  which  are  easy  to  implement,  yet 
not  too  inferior  to  optimal  ones  has  attracted  the  atten- 
tion of  several  investigators. 

Schoenberger  [7]  has  given  solutions  to  a  class  of 
problems  in  which  the  initial  conditions  are  known.   He  has 


also  presented  a  procedure  for  minimizing  the  expected 
value  of  the  performance  measure  when  the  probability  dis- 
tribution of  the  initial  conditions  is  known. 

Meditch  [8]  has  proposed  an  approximate  method  of  de- 
coupling complex  systems  which  simplifies  the  computation 
of  the  time-varying  feedback  gains. 

Koivuniemi  [9]  has  specified  the  form  of  the  feedback 
control  and  has  determined  the  unknown  parameters  to  mini- 
mize the  maximum  degradation  of  the  system  performance  over 
the  admissible  initial  states. 

Rekasius  [lo]  has  proposed  a  control  law  that  minimizes 
the  maximum  (with  respect  to  all  initial  states)  relative 
deviation  in  the  value  of  the  performance  measure  with  re- 
spect to  the  optimal  performance  measure.   Although  his 
method  avoids  some  of  the  difficulties  related  to  minimax 
problems,  it  is  impractical  for  systems  of  higher  than  the 
second  order. 

In  Kleinman  and  Athans'  [ll]  formulation  of  the  sub- 
optimal  linear  regulator  problem,  easily  realizable  time 
functions  are  used  to  generate  the  suboptimal  time-varying 
feedback  gains.   The  unknown  parameters  are  determined  to 
minimize  the  expected  value  of  the  performance  measure  over 
the  admissible  initial  states,  which  are  assumed  to  be  uni- 
formly distributed  on  a  unit  hypersphere. 

Kleinman,  Fortmann  and  Athans  [12]  have  given  an  ap- 
plication of  Kleinman  and  Athans'  [ll]  approach.   They 
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have  presented  an  algorithm  for  the  numerical  solution  of 
the  problem,  for, the  case  where  the  system  is  time-invari- 
ant and  the  suboptimal  feedback  gains  are  constrained  to  be 
piecewise  constant. 

Levine  and  Athans  [13]  have  also  used  an  average  per- 
formance measure,  but  they  have  let  the  control  vector  to 
be  a  time-varying  function  of  the  system'  output  rather  than 
the  system  state. 

Salmon  [14]  has  proposed  an  algorithm  which  can  be  ap- 
plied to  the  design  of  controllers  for  systems  with  unknown 
parameters,  including  initial  conditions.   In  this  method, 
the  controller  parameters  are  found  in  such  a  way  that  the 
maximum  of  either  the  system  performance  measure  or  the 
so-called  performance  sensitivity  with  respect  to  the  un- 
known system  parameters  is  minimized.   The  performance  sen- 
sitivity is  defined  as  the  percentage  or  the  absolute  in- 
crease in  the  performance  measure  from  that  of  an  ideal  con- 
troller caused  by  the  unknown  system  parameters.   The  ideal 
controller  is  defined  as  the  controller  which  is  capable  of 
measuring  unknown  system  parameters  and  generating  its  own 
parameters  accordingly. 

In  this  thesis,  after  a  brief  review  of  the  theory  of 
optimal  control  of  linear  regulators,  the  need  for  an  aux- 
iliary performance  measure  for  the  formulation  of  the  con- 
strained optimal  control  problem  is  shown  and  several  aux- 
iliary performance  measures  are  compared  using  a  simple 
second-order  example.   Then  the  constrained  optimal  control 


problem  is  formulated  and  a  numerical  method  of  solution 
for  the  case  of  constant  feedback  gains  is  given.  Next,  an 
approximate,  but  very  practical,  solution  to  the  problem 
for  the  case  of  piecewise-cons tant  feedback  gains  is  pro- 
posed.  Numerical  examples  that  illustrate  the  properties 
of  the  techniques  are  given. 
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II.   OPTIMAL  CONTROL  OF  LINEAR  REGULATORS 

Consider  the  linear,  time-varying  system 

x(t)  =  A(t)x(t)  +  B(t)u(t)  (1) 

y_(t)  =  C(t)x(t)  (2) 

where  x(t)  i-s  the  n-dimensional  state  vector,  u.(t)  is  the 
m-dimensional  unconstrained  control  vector,  y_(t)  is  the 
r-dimensional  output  vector,  and  A(t),  B(t)  and  C(t)  are 
time-varying  matrices  of  dimensions  n  x  n,  n  x  m,  and  r  x  n, 
respectively.   Determine  the  control  u * ( • )  that  minimizes 
the  performance  measure 


J(2i0*to,u(.))  =  \   xT(tf)Hx(tf) 


♦! 


,fcf 


[xT(t)Ci(t)x(t)  +  uT(t)R(t)u(t)  ]dt       (3) 
t 


where  t   and  t,.  >  t   are  the  initial  and  final  times,  re- 
o      f    o 

spectively,  x   =  x(t  )  is  the  initial  state,  H  is  a  real 
symmetric  positive  semi-definite  matrix,  Q(t)    is  a  time- 
varying  real  symmetric  positive  semi-definite  matrix,  R(t) 
is  a  time-varying  real  symmetric  positive  definite  matrix, 
and  the  final  state  x(t.p)  is  free. 
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The  solution  to  this  problem  is  well-known  [1,2,3,4], 
The  optimal  control  exists  and  is  given  by 

u*(t)  =  -  R"1(t)BT(t)K(t)x(t) 

*  -  F*(t)x(t)  (4) 

where  K(t)  is  the  unique  symmetric  positive  definite  solu- 
tion of  a  matrix  differential  equation  of  the  Riccati  type 
given  by 

K(t)  =  -  K(t)A(t)  -  AT(t)K(t)  -  &(t) 

+  K(t)B(t)R"1(t)BT(t)K(t)  (5) 

with  the  boundary  condition 

K(tf)  =  H  .  (6) 

The  state  of  the  optimal  system  is  the  solution  of 

x(t)  =  [A(t)  -  B(t)F*(t)]x(t)  (7) 


with  the  boundary  condition  x(t  )  =  x  .   The  optimal  per- 
formance measure  is  given  by 


j*<So.t0)  =  ^v^i-'i 


u(.)  =  u*(.) 


=  ^mt0)^.  <8) 
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The  matrix  K(t)  has  another  interesting  property.   For  ar- 
bitrary t  ^  tf  and  x(t) 

J*(x(t),t)  =  \   xT(t)K(t)x(t)  (9) 

Some  comments  on  the  results  just  given  are  in  order: 
first,  the  optimal  control  is  a  linear,  time-varying  feed- 
back of  the  system  states.   (See  Fig.  1) .   Second,  the 

matrix  Riccati  differential  equation,  Eq.  (5) ,  is  a  set  of 

2 
n  coupled,  first-order,  nonlinear  differential  equations 

which  can  be  solved  numerically  beginning  at  t  =  t,  and  in- 
tegrating backward  in  time  to  t  =  t  using  the  boundary  con- 
dition (6).   Actually,  only  n(n+l)/2  of  these  equations 
must  be  solved  because  K(t)  is  symmetric. 

In  implementing  the  controller  block  in  Fig.  1,  the 
most  appealing  scheme  is  to  calculate  K(t  )  by  integrating 
(5)  backward  off-line  and  then  to  calculate  F*(t)  on-line 
by  integrating  (5)  in  the  forward  direction  using  previous- 
ly calculated  K(t  ).   However,  this  is  not  possible  in  the 
time-varying  case  because  of  the  instability  of  (5)  in  the 
forward  time  direction  [2].   Even  if  the  system  is  time- 
invariant,  on-line  integration  is  not  always  desirable  due 
to  increased  complexity  of*  the  system.   Another  possibility 
is  to  provide  storage  for  the  elements  of  the  feedback  gain 
matrix  F* ( t)  and  to  store  the  precalculated  values.   This 
is  possible  because  the  elements  of  F_*(t)  do  not  depend  on 
the  system  state  but  depend  only  on  time;   therefore, 
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Fig.  1.   Linear  regulator  with  optimal  control. 

a  priori  knowledge  of  the  initial  state  is  not  necessary. 
However,  this  scheme  suffers  from  the  same  drawback  as  the 
previous  scheme,  namely,  increased  complexity. 

Suppose  that  F  (t)  is  readily  available.   Even  then 
the  realization  of  the  optimal  control,  u*(t),  requires  com- 
plete knowledge  of  the  system  state  x(t)  .   Reconstructing  or 
estimating  the  state  vector  [5,6]  from  the  knowledge  of  the 
output,  x(t)/  maY  a<3d  additional  complexity  to  the  control- 
ler. 

In  many  instances  it  is  possible  to  decrease  the  com- 
plexity of  a  control  system  considerably  by  constraining 
the  controller  configuration  and  allowing  a  small  increase 
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in  the  system  performance  measure  with  respect  to  optimal. 
If  the  adjustable  parameters  of  such  a  controller  are  de- 
termined in  some  optimal  fashion,  then  the  control  signal 
is  called  a  constrained  optimal  control.   In  the  following 
sections  the  design  of  constrained  optimal  controls  for 
systems  which  can  be  modeled  in  the  form  of  linear  regula- 
tors will  be  discussed. 
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III.   FORMULATION  OF  THE  CONSTRAINED 
OPTIMAL  CONTROL  PROBLEM 

For  the  system  and  the  performance  measure  given  by 
(1) ,  (2)  and  (3)  consider  the  control  law 

u(t)  =  -  P(t)y_(t)  (10) 

where  .P(t)  is  an  m  x  r   matrix  which  will  be  called  the  out- 
put-feedback gain  matrix.   The  physical  meaning  of  this  con- 
trol is  as  follows:   the  control  signal  is  generated  from 
the  system  output  vector  rather  than  from  the  system  state 
vector  as  in  the  optimal  control.   Substituting  (2)  in  (10) , 
one  obtains 

u(t)  =  -  P(t)C(t)x(t) 

i  -  F(t)x(t)  (11) 

The  value  of  the  performance  measure  for  the  control  law 
(11)  is  defined  as 


J(x,t.P(.))  =  J(x  ,t_,u(.))  (12) 

°°  °   °      lu(-)  =  -PCOZ(-) 

If  F(t)  defined  by  (11)  is  not  equal  to  F*(t)  defined  by 
(4)  for  all  t   ^  t  ^  tf/  then  the  control  (10)  is  not  op- 
timal.  In  general, 


J(2o.t0,P(.))  s  J*(2£0,tQ)  (13) 
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It  is  shown  in  the  Appendix  that 

J(x  ,t  ,P(-))  =  ^xTV(t  ,P(-))x   ,  (14) 

v— o   o  —       2  — o  —  o  —    — o 

where  V(t)   is  the  symmetric,  positive  definite  solution  of 
the  matrix  differential  equation 

V(t)  =-V(t)[A(t)  -  B(t)P(t)C(t)] 

-[A(t)  -  B(t)P(t)C(t)]T  V(t) 

-  £(t)  -  CT(t)PT(t)R(t)P(t)C(t)  (15) 

with  the  boundary  condition 

Y(tf)  =  H  (16) 

From  the  point  of  view  of  implementation,  the  control  (10) 
has  distinct  advantages  over  the  optimal  control:   first, 
knowledge  of  the  system  state  vector  x(t)  is  not  needed. 
Second,  constraints  may  be  imposed  on  P_(t)  to  make  the 
structure  of  the  control  simple,  e.g.,  P_(t)  may  be  required 
to  be  constant  or  piecewise  constant. 

A.    NEED  FOR  AN  AUXILIARY  PERFORMANCE  MEASURE 

It  is  reasonable  to  suggest  that  the  elements  of  P_(t) 
be  chosen  so  that  the  performance  measure  (12)  is  minimized, 


1 

Note  that  V(t)  depends  on  the  values  that  P(t)  attains  in 
the  interval  between  t  and  tf  .  However  this  will  be  shown 
explicitly  only  when  t  =  t   as  in  Eq.  (14) . 
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and  at  the  same  time  the  constraints  imposed  on  the  elements 
of  P(t)  are  satisfied.   However,  there  is  one  major  diffi- 
culty in  proceeding  in  this  fashion:   it  is  the  need  for  a 
priori  knowledge  of  the  initial  state  of  the  system.   It 
was  pointed  out  in  the  previous  section  that  although 
J  (x  , t  )  is  a  function  of  the  initial  state,  the  optimal 
feedback  gain  matrix  F*(t)  is  independent  of  the  initial 
state.   However,  this  is  not  the  case  with  the  control  (10) , 
in  general.   Not  only  J(x  ,t  ,P(-)),  but  also  the  best  out- 
put-feedback gain  matrix,  P(t),  depends  on  the  initial  state. 
If  the  initial  state  is  known  a  priori,  then  it  is  possible 
to  find  optimal  values  for  the  elements  of  P_(t),  for 
t   s  t  ^  tf,  satisfying  constraints  of  whatever  complexity 
that  may  have  been  imposed  on  them,  by  minimizing  the  per- 
formance measure  at  least  numerically,  if  not  analytically. 
If  the  initial  state  is  not  known,  then  one  approach  is  to 
define  an  auxiliary  performance  measure  which  is  independent 
of  the  initial  state.   Some  concepts  that  will  be  needed  in 
defining  an  appropriate  auxiliary  performance  measure  will 
be  given  first. 

The  Absolute  Degradation,  denoted  by  AD,  will  be 
defined  as 

AD(x  ,t  ,P(-))  =  J(x  ,t  ,P(.))  -  J*(x  ,t  )  ,       (17) 
— o   o  —         ~~o   o  —  o   o 

that  is,  the  absolute  degradation  is  the  increase  in  the 
performance  measure  with  respect  to  the  optimal  value  caused 
by  application  of  a  control  given  by  (10) . 


18 


The  Relative  Degradation,  denoted  by  RD,  is  given  by 


J(x  ,±  ,P(-) )  -  J*(x  , t  ) 

*b(2o. Vp(.»  *  ~°  y{K  -t  j    -°  °  («) 

— o  o 


Both  the  absolute  and  relative  degradations  are  always  non- 
negative  quantities. 

Assuming  that  the  possible  initial  states  may  lie 
within  a  hyperspherical  region,  with  center  at  the  origin 
and  having  a  specified  radius,  there  is  no  loss  of  general- 
ity in  further  assuming  that  the  set  of  admissible  initial 
states,  denoted  by  X,  consists  of  all  points  which  lie  on 
the  unit  hypersphere.   This  is  because,  for  any  admissible 
x   and  any  p  ^  0  , 

J(px  ,t  ,u(-))  =  i  p2xTK(t  )x 
Ir— o   o  —       2   — o  —  o  — o 

=  p2  J(x  ,t  ,u(-))  ,  (19) 

— o   o  — 

i.e.,  the  performance  measure  due  to  an  initial  state  which 
is  not  on  the  unit  hypersphere  is  a  constant  times  the  per- 
formance measure  due  to  the  corresponding  point  on  the 
unit  hypersphere. 

There  are  two  basic  approaches  in  choosing  an  auxiliary 
performance  measure:   one  is  the  statistical  approach,  the 
other  is  the  worst  initial  state  approach.   In  the  first  ap- 
proach, it  is  assumed  that  the  probability  distribution  of 
the  initial  state  is  known,  and  a  statistical  function  of 
the  performance  measure  that  is  independent  of  the  initial 
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state  is  taken  as  the  auxiliary  performance  measure.  Several 
authors  [7,11,12,13]  use  the  expected  value  of  the  perform- 
ance measure  over  the  admissible  initial  states  as  the  aux- 
iliary performance  measure.   Normally,  a  simple  probability 
density  function,  such  as  a  uniform  distribution  is  assumed. 
This  makes  the  expected  value  of  the  performance  measure  a 
very  simple  function  of  the  V  matrix  found  from  (15) . 
Several  variations  on  this  approach  depending  upon  the  con- 
straints imposed  on  the  structure  of  the  control  and  the 
assumed  admissible  initial  states  are  given  in  the  references 
cited  above. 

In  the  worst  initial  state  approach  [9,10,14],  the  max- 
imum (worst)  over  the  admissible  initial  states  of  any  one 
of  three  basic  quantities  -  performance  measure,  absolute 
degradation,  and  relative  degradation  -  can  be  taken  as  the 
auxiliary  performance  measure.   Therefore,  there  are  four 
functionals  of  P_(«)  which  can  be  used  as  the  auxiliary  per- 
formance measure.   These  are: 

J  (P(.))  =   E  {J(x,t  ,P(.))3  (20) 

1        x  ex 
— o 

J2(P(.))  =  Max  J(xQ,to,P(.))  (21) 

x  eX 
— o 

J  (P(-))  =  Max  AD(xQ,to,P(.))  (22) 

x  CX 
— o 

J4(P(.))  =  Max  RD(xo,tQ,P(.))  (23) 

X  6  X 
— o 
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The  output -feedback  gain  matrix,  which  minimizes  the 
auxiliary  performance  measure  is  called  the  constrained  op- 
timal output-feedback  gain  matrix  and  is  denoted  by  P  (t). 

The  constrained  optimal  control,  denoted  by  u_  (t),  is  given 

s 

by  (10)  with  P(t)  replaced  by  P  (t). 

The  performance  measure  resulting  from  the  application 

of  a  constrained  optimal  control  is  called  the  constrained 

optimal  performance  measure  and  is  denoted  by  J  (x  , t  ) . 
*  e  2      s  — o   o 


B.    A  COMPARISON  OF  SEVERAL  AUXILIARY  PERFORMANCE  MEASURES 
1.   Example  1 

The  purpose  of  this  example  is  to  illustrate  the 
differences  that  result  from  the  use  of  various  auxiliary 
performance  measures.   The  method  of  obtaining  the  solutions 
will  not  be  discussed,  but  rather  the  implications  of  each 
different  result  will  be  indicated. 

The  system  to  be  controlled  is  described  by 


x1(t) 


x2(t) 


n 


0 


x]_(t) 


_J  L- 


x2(t) 


u(t) 


(24) 


y1(t) 
y2(t) 


1     0 
0     1 


xx(t) 
x2(t) 


(25) 


The  performance  measure  for  optimal  control  is 

2 


J(xq,u(.))  = 


Cx12(t)  +  u2(t)]dt  . 


(26) 
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The  constrained  optimal  control  is  to  be  in  the  form 


Us(t)  =  "  [pll  P12] 


yx(t) 

y2(t) 


[Pll  P12] 


x1(t) 


x2(t) 


(27) 


where  p, ,  and  p..  «  are  constants  that  minimize  the  auxiliary 
performance  measure. 

This  problem  was  solved  four  times,  using  the  four  dif- 
ferent auxiliary  performance  measures  given  by  (20)  -  (23) . 
The  values  obtained  for  p.. ,  and  p  _  for  each  case  are  tabu- 
lated in  Table  1.   The  conclusion  that  can  be  drawn  looking 
at  that  table  is  very  important:   The  values  obtained  for 
the  parameters  of  the  constrained  optimal  control  depend 
very  much  on  the  auxiliary  performance  measure  used. 
Therefore,  the  proper  choice  of  an  auxiliary  performance 
measure  is  very  significant.   The  values  that  the  several 
performance  quantities  attain  as  functions  of  the  initial 
state  are  also  different  in  each  case.   Since  the  system  is 
only  second  order  and  the  initial  state  is  assumed  to  lie 
on  the  unit  circle,  just  one  quantity,  namely  the  ratio  of 
the  two  components  of  the  initial  state  vec tor ,  x2(  t  )/x,(t  )  , 
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Auxiliary  Per- 
formance Measure 

Pll 

P12 

Max  RD(x  , t  ,P) 

X  €X 

— o 

0.3055 

0.5932 

X  €   X 

— O 

0.8096 

1.1668 

Max   J(x  ,t  ,P) 

X  €  X 
— O 

0.7312 

1.5375 

Max  AD(x  ,  t  ,  P) 

x  CX 
— o 

0.5643 

0.9463 

Table  1.   Constrained  optimal  feedback 
gains  for  different  auxiliary- 
performance  measures  used  in 
Ex.1. 


is  enough  to  specify  a  particular  initial  state  .   The  op- 
timal performance  measure  J*,  constrained  optimal  perform- 
ance measure  J  ,  absolute  degradation  AD,  and  relative  de- 
gradation RD  are  plotted  as  functions  of  x0 ( t  )/x.(t  )  on 
^  e  2      o "  1   o 

one  graph  for  each  of  the  auxiliary  performance  measures  in 
Figs.  2,  3,  4  and  5. 


It  does  not  make  any  difference  whether  X2(tQ)  and  x-j_(t0) 
are  both  in  the  first  quadrant  or  both  in  the  third  quad- 
rant (similarly  for  the  second  and  fourth  quadrants)  due  to 
symmetry. 
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Fig.  2.   Various  performance  values  as 
functions  of  initial  state. 
(Maximum  relative  degradation 
minimized. ) 
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Various  performance  values  as 
functions  of  initial  state. 
(Expected  value  of  performance 
measure  minimized.) 
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Various  performance  values  as 
functions  of  initial  state. 
(Maximum  performance  measure 
minimized. ) 


Fig.  5.   Various  performance  values  as 
functions  of  initial  state. 
(Maximum  absolute  degradation 
minimized. ) 
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For  this  example,  it  is  clear  that  using  the  maximum 
relative  degradation  as  the  auxiliary  performance  measure 
(Fig.  1)  is  not  very  desirable.   Although  the  maximum  RD 
is  less  than  0.2  it  amounts  to  a  large  absolute  degradation 
for  initial  states  where  the  optimal  performance  measure  is 
already  high. 

When  it  is  assumed  that  the  initial  states  are  uniform- 
ly distributed  on  the  unit  hypersphere  and  the  expected 
value  of  the  performance  measure  is  used  as  the  auxiliary 
performance  measure  (Fig.  3)  the  relative  degradation  goes 
as  high  as  0.43.   But  this  occurs  for  initial  states  where 
the  optimal  performance  measure  is  quite  low,  and  the  ab- 
solute degradation  does  not  amount  to  a  large  value.   In 

this  case  the  J   curve  is  closer  to  the  J*  curve  on  the 
s 

average  than  the  other  three  cases.   But,  unless  separately 
calculated  as  in  this  example,  nothing  can  be  said  about 
maximum  absolute  degradation  which  may  be  more  than  allow- 
able. 

Fig.  4  shows  the  case  when  the  maximum  J  is  used  as 
the  auxiliary  performance  measure.   In  this  case  the  con- 
strained optimal  control  is  almost  as  good  as  the  optimal 
control  when  the  constrained  optimal  performance  measure  is 
the  maximum.   In  other  words,  the  control  is  the  best 
(closest  to  the  optimal)  when  the  initial  condition  is  the 
"worst".   This  is  a  very  desirable  feature.   However,  it 
results  in  a  very  pessimistic  design,  i.e.,  the  absolute 
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degradation  for  the  other  initial  states  becomes  quite 
high.   As  far  as  the  absolute  degradation  is  concerned,  al- 
lowing a  small  increase  for  the  worst  initial  state  may  re- 
sult in  a   significant  improvement  for  the  other  initial 
states,  as  is  the  case  in  Fig.  5. 

In  the  last  case  where  the  maximum  absolute  degrada- 
tion is  used  as  the  auxiliary  performance  measure  (Fig.  5) , 
the  constrained  optimal  performance  measure  as  a  function 
of  the  initial  state  is  made  as  close  to  the  optimal  per- 
formance measure  as  possible  by  minimizing  the  maximum  dif- 
ference between  the  two,  i.e.,  the  maximum  absolute  degra- 
dation.  When  this  approach  is  used  one  is  assured  that  the 
system  performance  will  not  be  degraded  more  than  a  certain 
amount  with  respect  to  the  optimal  system  performance  no 
matter  what  the  initial  state  is. 

2.   Some  Comments  on  Example  1 

Example  1  has  demonstrated  that  the  choice  of  an  ap- 
propriate auxiliary  performance  measure  is  a  significant 
step  in  the  design  of  constrained  optimal  controls  for 
linear  regulators.   It  is  easy  to  reject  the  maximum  rela- 
tive degradation  as  the  auxiliary  performance  measure  in 
almost  all  applications.   After  all,  one  would  not  want  to 
get  farther  away  from  the  optimal  performance  as  the  con- 
trol task  becomes  more  demanding.   The  remaining  three  de- 
finitions for  the  auxiliary  performance  measure  are  all 
reasonable  and  each  one  of  them  can  be  used  depending  on 
the  application.   During  the  remainder  of  this  work,  the 
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maximum  absolute  degradation  will  be  used  as  the  auxiliary 
performance  measure.   The  results  obtained  can  be  easily 
applied  to  the  case  where  the  maximum  performance  measure 
is  used  as  the  auxiliary  performance  measure.   The  case 
where  the  expected  value  of  the  performance  measure  is  used 
as  the  auxiliary  performance  measure  is  easier  to  handle 
mathematically  (especially  when  the  probability  distribu- 
tion of  the  initial  state  is  assumed  to  be  uniform  on  the 
unit  hypersphere)  and  is  treated  elsewhere  [7,11,12]  . 

C.    STATEMENT  OF  THE  CONSTRAINED  OPTIMAL  CONTROL  PROBLEM 

It  is  assumed  that  the  system  is  described  by  the  state 
and  output  equations 

x(t)  =  A(t)x(t)  +  B(t)u(t)  (1) 

yjt)  =  C(t)x(t)  (2) 

given  previously.   The  performance  measure  for  optimal  con- 
trol is 


J(2£0>t0/u(*))  =  \   2iT(tf)H  x(tf) 


♦■* 


t 


f 
[xT(t)Ojt)x(t)  +  uT(t)R(t)u(t)  ]dt  .      (3) 


Let  the  set  of  admissible  initial  states  be  X,  and  let  X 
consist  of  the  points  which  lie  on  the  unit  hypersphere. 
Let  CI   denote  a  specified  class  of  real  time  functions. 
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The  constrained  optimal  control  problem  is  to  find  a 
control 

Soft)  =  "  L  (t)z(t)  (28) 

such  that  the  elements  of  P  (t)  are  members  of  Q  and  the 

— s 

auxiliary  performance  measure 

J  (£(•))  =   Max  AD(x , t  ,P(-))  (29) 

a  ,  ,,     o   o  — 

x  €  X 
— o 

is  minimized. 

The  value  of  the  auxiliary  performance  measure  corre- 
sponding to  P  (t)  is  denoted  by  J   ,  i.e., 
s  a 


J*  =  Ja(P(.)> 


P(.)  =  Ps(.) 


=  Min   J  (P(-))  .  (30) 

P(-)cO  a 


The  magnitude  of  J   is  an  indication  of  how  close  the  con- 
strained optimal  control  is  to  the  optimal  control.   How- 
ever, a  normalized  number  is  usually  more  meaningful  in 
making  comparisons;   therefore,  the  Normalized  Degradation 
of  the  system,  denoted  by  ND,  will  be  defined  as 


J* 

ND   = .  (31) 

Max  J*(x  •  t  ) 

x  €X 
— o 
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ND  is  zero  if  the  performance  of  the  constrained  optimal 
control  is  the  same  as  the  performance  of  the  optimal  con- 
trol for  all  initial  states;   otherwise,  it  will  be  a  posi- 
tive number. 
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IV.   SOLUTION  OF  THE  CONSTRAINED 
OPTIMAL  CONTROL  PROBLEM 

A.    CONSTANT  OUTPUT-FEEDBACK  GAIN  MATRIX 

In  this  section,  the  constrained  optimal  control  prob- 
lem will  be  solved  for  the  case  where  the  elements  of  P_(t) 

are  constrained  to  be  constant  in  the  interval  between  t 

o 

and  t,.  . 

1.   Development  of  the  Method 


It  is  desired  that  the  constant  matrix  P   be  found 

— s 


such  that 


J  (P  )  =  J* 
a  v— s     a 


=  Min   Max   AD(x  ,  t  ,P)  .  (32) 

P  x  ex    -o  o  - 
—  — o 


This  is  a  minimax  problem,  and  there  are  many  difficulties 
associated  with  the  solution  of  such  problems:   first,  in 
general,  an  iterative  numerical  procedure  in  which  a  com- 
plete maximization  process  is  required  for  each  minimiza- 
tion step  is  necessary  [9];   second,  no  matter  how  smooth 

the  function  AD(x  ,P)  is  in  both  x   and  P,  the  function 

x— o  —  - o     — 

J  (P)  =   Max  AD(x  ,t  ,P)  (33) 

a  —       ^  „  o   o 

x  €X 
— o 

is  not,  in  general,  dif ferentiable  on  the  elements  of  P_. 
In  particular,  it  may  not  be  dif ferentiable  in  P  at  the 
point  yielding  the  minimum  [14,15].   Therefore,  some 
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method  which  does  not  require  the  partial  derivatives  of 
J  (P_)  with  respect  to  the  elements  of  P  is  more  suitable 
than,  for  example,  a  method  in  which  the  gradient  vector  is 
needed  and  the  partial  derivatives  are  evaluated  by  making 
small  perturbations. 

It  is  possible  to  simplify  the  procedure  consider- 
ably by  noting  that 


AD^.t^.P)  =J(xo,to.P)  -j'fx^) 

=  YXoT[v(t0,P)  -  K(t  )]x^ 


•£j£w<to.*>*0  (34) 


is  a  real  quadratic  form  in  x  ,  and  that  the  following 
well-known  [16,17]  theorem  about  the  extremal  properties 
of  the  eigenvalues  of  a  real  quadratic  form  is  applicable. 

Theorem:   The  global  maximum  of  a  real  quadratic 
form  on  the  unit  hypersphere  is  equal  to  the  largest  eigen- 
value of  the  quadratic  form,  and  moreover  the  corresponding 
eigenvector  is  the  vector  drawn  from  the  origin  to  the 
point  on  the  hypersphere  where  the  quadratic  form  achieves 
its  maximum.   Therefore 

Max  AD(xQ,to,P)  =  \   X  (P)  (35) 

x  ex 
— o 

where   Xn  (P.)  is  the  largest  eigenvalue  of  W(t  ,P_)  defined 
by  (34) ,  and  the  problem  is  to  minimize   X, (P)  with  respect 
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to  the  elements  of  P_  ;   that  is,  to  find  the  constant  matrix 

P   such  that 
— s 

J  (P  )  =  J* 
a  v— s      a 

=  |  Min  X, (P)  .  (36) 

P    L 


Such  a  formulation  has  several  advantages  over  the  formula- 
tion given  by  (32) .   First, 

X,  (P)  =  2   Max  AD(x  ,t  ,P)  (37) 

1  —         ^  „    — o  o  — 
x  e  X 
— o 

can  be  evaluated  without  the  need  to  find  the  point  where 
the  maximum  occurs.   This  is  very  important  because  (37)  is 
evaluated  many  times  during  the  minimization  process,  and 
any  savings  in  the  number  of  necessary  computations  is  sig- 
nificant.  Second,  the  programming  required  is  extremely 
simple  because  very  efficient  methods  are  available  for 
finding  the  largest  eigenvalue  of  a  real  symmetric  matrix, 
and  many  computer  facilities  have  several  standard  subrou- 
tines that  can  be  used  for  this  purpose.   However,  it  should 
be  noted  that  a  subroutine  that  finds  all  eigenvalues  (and 
eigenvectors)  of  a  general  square  matrix  is  not  as  suitable 
for  this  purpose  as  one  which  finds  only  the  largest  eigen- 
value of  a  real  symmetric  matrix. 
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2.   Algorithm  for  Computing  the  Constant  Output- 
Feedback  Gain  Matrix  -  Algorithm  I 

a.  Calculate  K(t  )  by  integrating  (5)  backward 
with  the  boundary  condition  (6)  . 

b.  Guess  starting  values,  for  the  entries  of  P_. 

c.  Find  P  that  minimizes  X. (P)  by  using  a  sub- 
routine that  preferably  does  not  require  the  evaluation  of 
the  gradient.   Such  a  subroutine  will,  in  general,  require 
the  evaluation  of  X..  (P_)  many  times.   This  can  be  accomplish- 
ed as  follows:- 

(1)  Calculate  V(t  ,P)  by  integrating  (15)  back- 
ward with  the  boundary  condition  (16) ,  using  the  current 
value  of  P. 

(2)  Calculate  W(t  ,P)  using  (34). 

(3)  Find  the  largest  eigenvalue,  X.  (jP)  ,  of 
W(t  ,P)  using  a  suitable  subroutine. 

d.  If  the  location  of  the  point  where  the  worst 
degradation  occurs  is  desired,  then  find  the  eigenvector 
corresponding  to  X..  (P_  )  . 

In  applying  this  algorithm  to  the  examples  given 
in  the  rest  of  this  thesis,  the  minimizations  were  accom- 
plished using  a  subroutine   that  performs  the  pattern 
search  method  of  Hooke  and  Jeeves  [18,19];   the  largest 


1 

Subroutine  DIRECT,  NPS  Computer  Facility. 
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eigenvalues  were  found  using  a  subroutine   that  utilizes 

Givens-Householder  method;   and  integrations  were  performed 

2 
using  a  Runge-Kutta-Gill  fourth-order  method  . 

3.   Application  of  Algorithm  I  -  Example  2 

The  purpose  of  this  example  is  to  illustrate  the 
application  of  the  described  algorithm  to  a  simple  second- 
order  system,  and  also  to  illustrate  the  importance  of  using 
the  appropriate  algorithm. 

The  system  to  be  controlled  is  described  by 


xx(t) 
x2(fc) 


0     1 
■3    -1 


xx(t) 

+ 

0 

_x2(t)_ 

1 

_    _ 

u(t) 


(38) 


yx(t) 

y2(t) 


1     0 
0     1 


x-^t) 


x2(t) 


(39) 


The  performance  measure  for  optimal  control  is 


J(Xo,u(.)) 


1 
2   J 


5ff 


[x12(t)  +  x22(t)  +  u2(t)]dt 


(40) 


The  constrained  optimal  control  is  to  be  in  the  form 


Us(t)  =  "  [Pll  Pl2_ 


Yx(t) 


y2(t) 


(41) 


Subroutine  GIVHQ,  see  Ralston  and  Wilf  [20 ] 


Function  RKLBEQ,  NPS  Computer  Facility. 
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where  p, ,  and  p1 2  are  constants.   Expanding  the  Riccati 
equation  (5)  with 


A  = 


0     1 


-3    -1 


.       B  = 


0 


,   Q  = 


1     0 


U 


0     1 


,  R  =  1 


one  obtains 


ki;L(t)  =  6k12(t)  -  1  +  k12(t) 


k12(t)  =  k12(t)  -  ki;L(t)  +  3k22(t)  +k12(t)k22(t)   (42) 
k22(t)  =  2[k22(t)  -  k12(t)J  -  1  +  k222(t) 


with  the  boundary  conditions 


11 


5ff 
4 


=  k 


12 


5ff 
4 


=  k 


22 


5ff 
4 


=   0 


(43) 


Note  that  due  to  the  symmetry  of  K(t)  only  three  equations 
are  needed.   Expanding  the  equation  (15)  with 


^  =  [Pll  P12]   a 


nd   C  = 


0 


one  obtains 


vxl(t)  =  2(3+p11)v12(t)  -  1  -  p 


11 


v12(t)  =  (l+p12)v12(t)-v11(t)+(3+p11) v22(t) "P11P12 


v22(t)  =  2(l+p12)v22(t)-2v12(t)  "1-P12 


(44) 
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with  boundary  conditions 


v11Cf)=v12(f)=v22(f 


Again  due  to  symmetry  only  three  equations  are  needed. 

The  proposed  algorithm  gives  the  solution  for  the 
constant  output- (or  state)  feedback  gain  matrix  as 

P   =  CO. 1537    0.5097] 
— s 

The  normalized  degradation  (ND)  is  0.00055  which  shows  that 
the  constrained  optimal  control  is  almost  as  good  as  the 
optimal  control. 

Koivuniemi  [9]  treats  the  same  system  as  an  ex- 
ample for  the  design  of  a  constrained  optimal  controller 
which  he  calls  a  specific  optimal  controller.   He  uses  the 
same   auxiliary  performance  measure  as  here,  but  obtains  a 
result  which  is  quite  different.   He  gives  the  constant  out- 
put-(or  state)  feedback  gain  matrix  as 

P   =  [0.69    0.76] 
— s 

Assuming  that  the  initial  states  lie  on  the  unit  circle, 

the  optimal  performance  measure  J*,  constrained  optimal 

performance  measure  J  ,  absolute  degradation  AD,  and  rela- 

tive  degradation  RD  are  plotted  as  functions  of  x2 ( t  )/x-(t  ) 

using  the  P   matrix  obtained  here  and  the  P   matrix  given 
J  —  s  — s 

in  [9]  in  Figs.  6  and  7  respectively,  for  comparison. 
Clearly  the  result  found  here  results  in  a  much  better 
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Various  performance  values  as 
functions  of  initial  state  -  using 
feedback  gains  found  in  Example  2. 


Fig.  7.   Various  performance  values  as 

functions  of  initial  state  -  using 
feedback  gains  given  in  Ref.  9. 
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performance  for  all  initial  states  than  the  result  given  by 
Koivuniemi.   This  is  also  obvious  from  a  comparison  of  the 
normalized  degradation  (ND)  figures.   It  is  0.057  for 
Koivuniemi 's  design  —  much  higher  than  0.00055  obtained 
here. 

This  example  has  illustrated  the  importance  of 
using  an  efficient  algorithm  for  the  solution  of  the  mini- 
max  problem.   Apparently,  the  algorithm  given  in  [9 J  has 
failed  at  some  point,  resulting  in  a  solution  which  is  not 
as  good  as  it  could  be  for  this  example. 

If  the  output  of  the  same  system  (38)  is  given  by 


y1(t)  =  [1    0] 


x1(t) 


x  (t) 


(46) 


and  the  constrained  optimal  control  is  specified  in  the 
form 


ug(t)  =  -  p11y1(t)  , 


(47) 


then  the  expansion  of  (15)  with 


P  =  pi;L    and   C  =  [l    0] 


gives 


vlx(t) 


=  2(3+p11)v12(t)  -1-P1X 


v12(t)=  v12(t)-  vi;L(t)  +  (3+p11)v22(t) 


(48) 


v22(t)=  2v?2(t)  -  2v,  2(t)  -  1 
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with  boundary  conditions  (45) .   Applying  the  same  algorithm, 
one  obtains 

P   =  p, ,  =  -0.236 
— s    ^11 

which  gives  a  normalized  degradation  (ND)  of  0.137. 
Koivuniemi's  result  for  this  case  is 

Ps  =  Pn  -  "0.25 

which  agrees  reasonably  well  with  the  result  found  here. 
Evidently,  the  algorithm  given  in  [9]  works  in  this  case. 

B.    PIECEWISE-CONSTANT  OUTPUT-FEEDBACK  GAIN  MATRIX 

In  this  section,  the  constrained  optimal  control  prob- 
lem will  be  solved  for  the  case  where  the  elements  of  JP(t) 
are  constrained  to  be  piecewise-constant  in  the  interval 

between  t   and  t_  . 
o       f 

1.   Development  of  the  Method 

Suppose  that  there  are  N  subintervals  of  time,  each 
of  duration  At,  during  which  the  elements  of  P_(t)  are  con- 
stant.  Let 

t.  =  t   +  iAt,    i  =  0,1,  ...  ,  N  (49) 

and  let 

P(t)  =  P ±    ,         t±    ^   t  <  ti+1  ,   i  =  0,1,  ...  ,  N-l 

(50) 
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Since  P.  is  an  m  x  r   matrix  and  since  there  are  N  such  con- 
stant matrices,  one  can  minimize  the  auxiliary  performance 
measure  given  by  (29)  with  respect  to  the  mrN  elements  which 

are  to  be  determined  and  obtain  the  piecewise-constant  P  (t). 

r  — s 

However  if  mrN  is  a  large  number,  the  minimization  of  the 
auxiliary  performance  measure  with  respect  to  such  a  large 
number  of  variables  may  be  a  very  difficult  and  time-con- 
suming task. 

A  much  simpler,  yet  possibly  more  powerful  and 
practical  approach  will  be  used  here.   This  approach  is  in- 
spired by  the  Principle  of  Optimality  of  Bellman  [l].   Sup- 
pose that  the  time  t„  ,  has  been  reached  and  there  remains 
*  N-l 

only  one  interval  during  which  control  can  be  applied.   At 
this  point,  if  x(t   ,)  were  known  a  priori,  then  to  deter- 
mine P.N_i  would  be  a  simple  parameter  optimization  problem, 
and  P^r-i  would  be  determined  to  minimize  J(x  ( t   ,  )  ,  tN_,,P_„_,). 
However,  x(t   -,)  is  not  known  a  priori,  therefore,  one  is 
again  confronted  with  the  problem  of  defining  an  auxiliary 
performance  measure.   Fortunately,  the  solution  proposed 
for  the  constant-P  case  in  the  previous  section  can  be  ap- 
plied to  this  case  also  by  simply  taking  tN_,  as  the  ini- 
tial time  and  applying  the  Algorithm  I „   This  gives  the 

elements  of  P..  ,  as  well  as  K(t.T  ,),  V(t.T  ,  )  and  W  ( t.T  ,). 

—N-l  — x  N-l    —  N-l       —  N-l 

This  solution  for  the  last  interval  is  equivalent  to  assum- 
ing that  the  admissible  x(t   ,)  lies  on  the  unit  hyper- 
sphere  (without  loss  of  generality)  and  minimizing  the  max- 
imum absolute  degradation  with  respect  to  the  admissible 
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x(t.T  ,).   Similarly,  P   0  can  be  determined  using  t   _  and 

INI  —  J.  IN  —  Z  N— 2 

t   ,  as  the  initial  and  the  final  times  respectively  and 
using  K(t„_,)  and  V(t„_,)  obtained  as  a  result  of  the  pre- 
vious application  as  the  boundary  conditions  in  performing 
the  integrations.   The  same  procedure  is  applied  to  all 

subintervals  until  P  ,  K(t  )  and  V(t  )  are  obtained  at  the 

— o  —  o      —  o 

last  application. 

2.   Algorithm  for  Computing  the  Piecewise-Cons tant 
Output-Feedback  Gain  Matrix  -  Algorithm  II 

a.  Let  i  =  N. 

b.  Let  tr  =  t.  and  t   =  t.  ,  . 

f     1       o     1-1 


c.      Apply  Algorithm   I    to   obtain   P_.    ,  ,    K(t._,)    and 


vu..,) 


d.  If   i   =    1,    go    to    f. 

e.  Let   i   =   i    -    1,    go    tob. 

f .  S  top . 

3.   Some  Comments  Concerning  the  Application  of  Algo- 
rithm II 

The  optimal  value  for  the  auxiliary  performance 
measure,  J   ,  obtained  by  applying  Algorithm  II  will  be 

cL 

greater  than  the  value  that  would  be  obtained  if  the  mini- 
mization were  accomplished  at  one  step  with  respect  to  mrN 
elements  as  indicated  earlier.   However,  it  should  be  re- 
membered that  in  the  latter  case  the  "worst"  initial  state 
(the  initial  state  that  causes  the  maximum  absolute  degrada- 
tion) is  assumed  at  time  t   only  and  not  at  the  beginning  of 
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each  subinterval,  whereas,  in  the  method  of  solution  pro- 
posed here,  the  constant  output -feedback  gain  matrix  for 
each  subinterval  is  determined  to  minimize  the  maximum  ab- 
solute degradation  at  the  beginning  of  the  subinterval. 

It  is  even  possible  that  the  normalized  degradation 
may  increase  with  a  small  increase  in  the  number  of  subin- 
tervals.   This  is  because  the  "worst"  is  assumed  to  occur 
more  often  and  the  parameter  values  that  are  chosen  with 
this  in  mind  may  result  in  higher  normalized  degradation 
than  the  case  where  "worst"  is  assumed  less  often  or  just  at 
the  beginning.   However,  when  this  occurs,  it  still  does  not 
mean  that  increasing  the  number  of  subintervals  is  not  ad- 
vantageous.  This  entirely  depends  on  the  application.   If 
the  system  to  be  controlled  is  in  an  environment  which  is 
known  very  well  and  no  disturbances  are  anticipated,  then 
increasing  the  number  of  subintervals  in  the  manner  proposed 
here  does  not  result  in  a  "better"  system,  unless  there  is 
a  decrease  in  normalized  degradation.   If  piecewise-constant 
output -feedback  gains  are  desired  in  such  a  case,  then  one 
should  perform  the  minimization  at  one  step  with  respect  to 
all  mrN  elements  as  mentioned  earlier.   However,  in  practice 
many  systems  operate  in  environments  where  unpredictable 
disturbances  can  occur  at  any  time.   In  those  cases,  increas- 
ing the  number  of  subintervals  even  when  the  normalized  de- 
gradation does  not  decrease  is  desirable,  at  least  from  a 
conservative  design  point  of  view. 
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When  all  of  the  states  of  a  system  are  available 
as  outputs  and  the  number  of  subintervals  approaches  infin- 
ity, i.e.,  the  constraints  are  relaxed,  then  one  expects 
the  constrained  optimal  control  to  approach  the  optimal  con- 
trol and  the  normalized  degradation  to  approach  zero.   This 
was  the  case  in  all  the  examples  studied  by  the  author. 
4.   Application  of  Algorithm  II 
a.   Example  3 

The  purpose  of  this  example  is  to  illustrate 
the  application  of  Algorithm  II  to  a  third-order  system. 
The  system   to  be  controlled  is  described  by 


xx(t) 

-l 

0 

0 

xx(t) 

2 

x2(t) 

= 

0 

0 

2 

x2(t) 

+ 

2 

x3(t) 

0 



-2 

0 

!_x3(t)_ 

-1 

y^t)" 

1 

0 

o 

~x1(t)~ 

y2(t) 

= 

0 

1 

i 
0 

1 

j    x2(t) 

• 

y3(t)_ 

0 

0 

i  ! 

x3(t) 

u(t) 


(51) 


(52) 


The  performance  measure  for  optimal  control  is 


J(xq,u(.))  = 


[x1(t)x2(t)x3(t)l 


1 

-1 

o~ 

xx(t) 

1 

1 

0 

x2(t) 

0 

0 

0 

_x3(t)_ 

This  system  is  treated  in  Ref.  12  where  constant  and 
piecewise-constant  gains  are  computed  using  a  different 
auxiliary  performance  measure. 
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The  constrained  optimal  control  is  to  be  in  the  form 


Ug(t)     =    "    [pi]L(t)     P12(t)     P13(t)] 


Yx(t) 

y2(t) 
y3(t) 


(54) 


where  [p-i-j(t)  p,  2(t)  p,-(t)]  is  a  piecewise-constant  out- 
put-feedback gain  matrix.   The  matrices  needed  for  the  ap- 
plication of  Algorithms  I  and  II  are: 


P"i   o   o 

c  =  !  o   l   o 

o   o   1 


r-i 

0 

0 

M 

A    =         0 

0 

2 

B    =|    2 

0 

-2 

0 

_-!_! 

~2 

-2 

0~ 

Q.  = 

-2 

2 

0 

R   =    2 

0 

0 

0 

p  =  [pu(t)  P12(t)  P13(t)] 


and  H  =  0.   This  problem  was  solved  for  the  one-subinterval 
(a  constant  feedback  gain  matrix)  case  using  Algorithm  I, 
and  for  two  and  ten-subinterval  (piecewise-constant  feed- 
back gain  matrices)  cases  using  Algorithm  II.   The  elements 
of  F  (t)  =  P  (t)C  and  F*(t)  =  R~  BTK(t)  are  plotted  in  Figs. 
8,  9,  and  10  for  each  case.   The  normalized  degradation  is 
0.0356  for  the  one-subinterval  case  and  decreases  to  the 
values  0.0135  and  0.0025  for  the  two  and  ten-subinterval 
cases,  respectively.   As  the  number  of  subintervals  is 
further  increased,  the  constrained  optimal  feedback  gains 
become  closer  to  the  optimal  feedback  gains,  and  the 
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Fig.  8.  Optimal  and  constrained  optimal 
feedback  gains  for  Example  3  -- 
One  subinterval. 
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Fig.  9.  Optimal  and  constrained  optimal 
feedback  gains  for  Example  3  — 
Two  subintervals. 
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Fig.  10.  Optimal  and  constrained  optimal 
feedback  gains  for  Example  3  — 
Ten  subintervals. 
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normalized  degradation  decreases.   When  the  number  of  sub- 
intervals  is  fifty  the  normalized  degradation  is  0.0002. 
Clearly,  for  this  example,  one  can  get  as  close  to  the  op- 
timal as  desired  by  increasing  the  number  of  subintervals. 
b.   Example  4 

The  purpose  of  this  example  is  to  investigate 
and  compare  the  effectiveness  of  using  different  states  of 
a  linear  regulator  to  form  the  feedback  control,  and  to 
provide  another  illustration  of  the  applications  of  Algo- 
rithms  I  and  II.   The  system  to  be  controlled  is  again 
given  by  (51)  and  the  performance  measure  for  the  optimal 
control  by  (53) .   This  time  the  output  is  a  scalar  given  by 


y(t)  =  c 


xx(t) 
x2(t) 
x3(t) 


(55) 


where  C_  is  a  1  x  3   matrix,  with  only  one  element  equal  to 
one  and  the  rest  equal  to  zero.   In  other  words,  only  one 
of  the  states  is  available  as  an  output.   The  constrained 
optimal  control  is  specified  in  the  form 


u  (t)  =  -  p(t)y(t) 


(56) 


where  p(t)  is  the  piecewise-constant  output-feedback  gain. 
Table  2  gives  the  normalized  degradations  for  different 
outputs  and  for  various  numbers  of  subintervals.   Clearly, 
C  =  Co   0   l]  results  in  the  best  constrained  optimal 
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control,  i.e.,  x  (t)  is  quite  effective  in  forming  the  con- 
trol.  Considering  that  the  normalized  degradation  is  0.5653 
without  any  control  (u(t)=  0)  it  can  be  seen  that  C  =  [l  0  O] 
or  C  =  [0  1  o]  provide  very  little  improvement  over  no  con- 
trol.  When  C  =  [O  0  l],  there  is  a  small  increase  in  the 
normalized  degradation  when  the  number  of  subintervals  is 
increased  from  one  to  two.   This  was  mentioned  as  a  possi- 
bility previously.   However,  further  increase  in  the  number 
of  subintervals  decreases  the  normalized  degradation  again. 


1 
G 

One 
Subinterval 

Two 
Subintervals 

Four 
Subintervals 

Li  o 

o] 

0.5022 

0.5022 

0.5022 

[0  1 

o3 

0.5252 

0.4767 

0.4419 

Co  o 

i] 

0.0907 

0.0942 

0.0822 

Table  2.   Normalized  degradations  for  different  outputs 
and  number  of  subintervals  for  Example  4. 


c .   Example  5 

In  this  example,  Algorithms  I  and  II  are  ap- 
plied to  a  second-order,  time-varying  system.  The  system 
is  described  by 
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xx(t) 


x2  ( t)J       0 


25-t    25-t 


L_ 


xx(t) 

■ 

+ 

x2  (  t)J 

u(t) 


(57) 


yx(t) 


y2(t) 


1     0 


0     1 


xx(t) 


x2(t) 


(58) 


The  performance  measure  for  optimal  control  is 

.10 
J(x_o,u(-))  =  |  ;   [200  x12(t)  +2u2(t)]dt. 


(59) 


The  constrained  optimal  control  is  to  be  in  the  form 


ug(t)  =  -  [p11(t)  p12(t)] 


Yx(t) 


y2(t) 


(60) 


where  [p, ,(t)  p,?(t)]  is  a  piecewise-constant  output-feed- 
back gain  matrix.   Expanding  the  equations  (5)  and  (15) 
with 


These  equations  represent  the  perturbed  motion  of  the 
flight  of  an  airplane  in  a  vertical  plane  on  a  gliding  path 
inside  the  equisignal  zone  of  the  glide  radio  beacon,  with 
constant  velocity.   For  more  detail  on  the  description  of 
the  system,  see  Example  1  of  Ref.  21  where  an  optimal  con- 
troller is  designed  for  the  system. 
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A   = 


-1 


25-t    25-t 


B   = 


0.08167 


C   = 


1     0 


0     1 


Q.    = 


200    0 


0     0_ 


R  =  2  and  H  =  0  and  applying  Algorithms  I  and  II ,  the  prob- 
lem was  solved  for  several  cases  with  various  numbers  of 

subintervals.   The  elements  of  F  (t)  =  P  (t)C  and 

— s      — s   — 

F*(t)  =  R_1BTK(t)  are  plotted  in  Figs.  11  and  12  for  one 
and  two-subinterval  cases  respectively.   The  normalized  de- 
gradation is  0.02  for  the  one-subinterval  case  and  0.0852 
for  the  two-subinterval  case.   This  increase  in  the  normal- 
ized degradation  with  an  increase  in  the  number  of  subin- 
tervals was  mentioned  previously  as  a  possibility  in  the 
comments  following  the  description  of  Algorithm  II.   How- 
ever, as  the  number  of  subintervals  is  increased  further, 
the  normalized  degradation  decreases  and  approaches  zero, 
e.g.,  it  is  0.0205  for  the  ten-subinterval  case  and  0.0015 
for  the  f if ty-subinterval  case.   This  example  shows  that 
the  application  of  Algorithms  I  and  II  to  time-varying  sys- 
tems is  as  straightforward  as  the  application  to  time- 
invariant  systems. 
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Fig.  11.   Optimal  and  constrained  optimal  feedback 
gains  for  Example  5  —  One  subinterval. 
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Fig.  12.   Optimal  and  constrained  optimal  feedback 
gains  for  Example  5  —  Two  subintervals. 


54 


V.   CONCLUSIONS 

A .    SUMMARY 

The  problem  of  designing  optimal  controls  for  linear 
regulators  under  complexity  constraints  has  been  investi- 
gated.  The  need  for  an  auxiliary  performance  measure  was 
demonstrated  and  several  candidates  for  the  auxiliary  per- 
formance measure  were  compared.    The  maximum  of  the  abso- 
lute degradation  over  the  admissible  initial  states  was  se- 
lected as  the  auxiliary  performance  measure.   A  method  was 
developed  and  a  computational  algorithm  (Algorithm  I)  was 
given  to  solve  the  resulting  minimax  problem  for  the  case 
where  the  feedback  gains  are  constrained  to  be  constant. 
A  conservative  engineering  approach  inspired  by  the  Princi- 
ple of  Optimality  was  used  for  the  solution  of  the  same 
problem  for  the  case  where  the  feedback  gains  are  constrain- 
ed to  be  piecewise-constant.   It  was  assumed  that  the  "worst" 
initial  state  could  occur  at  the  beginning  of  each  subin- 
terval  and  the  parameters  were  determined  accordingly.   A 
computational  algorithm  (Algorithm  II)  was  given  for  this 
purpose. 

Both  algorithms  were  programmed  in  FORTRAN  IV  for  the 
IBM  360  in  use  at  the  NPS  Computer  Facility.   Since  the 
algorithms  rely  heavily  upon  existing,  well-established 
library  subprograms,  the  programming  required  was  very 
simple,  yet  efficient  and  reliable.   In  none  of  the  problems 
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solved  did  the  results  depend  on  the  initial  guess  for  the 
parameter  values.   For  the  minimization  step  of  the  algo- 
rithms a  gradient  scheme  in  which  partial  derivatives  were 
evaluated  by  making  small  perturbations  was  also  tried  and 
gave  the  same  results  as  the  pattern  search  in  all  cases. 
The  gradient  algorithm,  however,  required  using  more  than 
one  initial  guess  in  some  instances,  and  did  not  improve 
the  computation  time  observed  with  the  pattern  search 
method. 

Algorithm  II  does  not  add  much  complexity  to  Algorithm 
I,  nor  does  it  increase  the  computation  time.  This  is  be- 
cause most  of  the  computation  time  is  used  for  integration, 
and  when  the  number  of  subintervals  is  increased,  there  is 
a  corresponding  decrease  in  the  number  of  integration  steps 
per  subinterval  and  hence  no  significant  change  in  computa- 
tion time  occurs. 

B.    PROBLEMS  FOR  FURTHER  STUDY 

It  would  be  worthwhile  to  investigate  the  application 
of  the  given  algorithms  to  linear  tracking  problems  because 
these  problems  have  wide  applications  and  methods  for  easi- 
ly realizable  controllers  for  such  systems  are  very  desir- 
able. 

Another  worthwhile  extension  is  the  application  of  the 
given  methods  to  the  design  of  simple  compensators  for 
automatic  control  systems.   For  example,  for  a  third-order 
linear  regulator  with  only  one  output,  a  second-order 
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compensator  could  be  combined  with  the  system  and  then  in 
addition  to  the  output  feedback  gain,  the  compensator  para- 
meters could  be  optimized  in  a  minimax  sense  for  the  over- 
all fifth-order  system. 
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APPENDIX 

DERIVATION1  OF  EQUATIONS  (14) ,(15)  AND  (16) 

For  the  system  and  the  performance  measure  given  by 
(1),  (2)  and  (3),  consider  the  control  law  given  by  (10)  or 
(11) .   Substituting  (11)  in  (1) ,  one  obtains 

x(t)  =  [A(t)  -  B(t)F(t)]x(t)  .  (A.l) 

Let  $  (t,t  )  be  the  transition  matrix  [l]  of  (A.l),  then 
the  motion  of  (A.l)  is  given  by 


x(t)  =  ^(t,^)^  (A. 2) 


and  <£  (t,t  )  satisfies  the  differential  equation 


^*g(t,to)  =  [A(t)  -  B(t)F(t)]^(t,tQ)  (A. 3) 


with  the  boundary  condition  $  ( t  ,  t  )  =  I.   In  addition, 

1  — s  o   o    — 

the  final  state  is  related  to  any  preceding  state  by 

x(tf)  =  ^(tf,t)x(t)  ,  (A. 4) 

and,  in  particular,  to  the  initial  state  by 

x(tf)  ^(tf,to)xQ  .  (A. 5) 

2 
^.s(tf,t)  satisfies   the  differential  equation 


This  derivation  follows  the  outline  given  in  [8], 

2 

This  can  be  shown  by  differentiating  (A. 4)  with  respect 

to  t  and  noting  that  dx(tf)/dt  =  0. 
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dUrSg^f't)  =  "^(tf  fc)tA(t)-B(t)F(t)]  (A. 6) 

for  all  tf  and  t,  with  the  boundary  condition  <f>  (t-,  t_)  =I_ 
Substituting  (11)  in  (3) f  one  obtains 

J(2£0'to,P(.))  =  |  xT(tf)H  x(tf) 

tf 

+  ^-J   xT(t)[^(t)+FT(t)R(t)F(t)]x(t)dt.         (A. 7) 
o 


Further  substitution  for  x(t)  an(3  ^(tf)  using  (A. 2)  and 
(A. 5)  gives 

J(x  ,t  ,P(.)  )  =lxT$T(L,t  )H  *  (t-,  t  )x 
v— o   o  — v       2  -^o  — s    f*  o sv  f   o  — o 

tf 

+  |  J  2£oT^T^t't0)^(t)+FT(t)R(t)F(t)]^(t;to)xodt. 
o 

(A. 8) 
Combining  the  two  terms  gives 

Jtx^t^M)  =  fx£  {  ^T(tf,to)H  ^(t£,to) 


+  j   ^T(t,to)[£(t)+FT(t)R(t)Fjt)]^(t,to)dt}xo  . 


o 

(A. 9) 


Defining 


Y(t)  =  ^T(tf,t)H  $Jtf,t) 


fcf 


+     $J(T,t)  [Q(T)+FT(T)R(T)F(T)]$  (T,t)dT,     (A. 10) 


f" 
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then  V(t  ,P(*))   is  the  term  in  the  braces  in  (A. 9)  and 

j(^,to,p(.n  =^xoTY(to,P(0)xo  .  d4) 

which  is  repeated  here  for  convenience,  is  obtained. 

2 
To  show  that  (15)  and  (16)  hold,  differentiate   (A. 10)  with 

respect  to  t,  with  the  result 

V(t)  =  $3T(tf,t)H  *,(tf,  t)  +  ^T(tf,t)H  i(tf,t) 


c   f 


+ 


{  i  T(T,t)[S(T)+FT(T)R(T)F(T)]$  (T,t) 


+  $   (T,  t)  [fi(T)+P  (T)R(T)F(T)  ]*  (T,  t)  \    dT 

-    [£(t)+FT(t)R(t)F(t)]  .  (A. 11) 


Substituting  (A. 6)  for  $  (tf,t)  and  $  (T,t)  (noting  the 
necessary  correspondence  between  tf  and  T  in  the  latter 
case)  gives 


See  the  footnote  on  page  17. 
tf 

2  If  I(t)  =    f(T,t)dT,  then  by  Leibnitz's  rule 

J  t 


t 
.  f 

dt      „t 


Ltl=   J  f  af^.t)  dT  _  f(tffc) 
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V(t)     =    -    [A(t)     -    B(t)F(t)]T^(tf,t)H    *g(tfft) 


-    %     Cfefrt)H    ^(tf,t)[A(t)     -    B(t)P(t)] 

[A(t)-B(t)F(t)  ]T$T(T,t)  [£(T)+FT(T)R(T)F(T)  ]*    (T,t)dT 

t  S  ~S 


,'f 


$T(T,t)  [£(T)+FT(t)R(T)F(T)]<I>    (t,  t)  [A  ( t)  -B  ( t)  P  ( t)3dT 


t    "* 


fi(t)     -    FT(t)R(t)F(t)     .  (A. 12) 


Combining  the  first  and  third  terms  and  similarly  the  second 
and  fourth  terms  of  (A. 12),  and  identifying  V(t)  as  given 
by  (A. 10)  one  obtains 


V(t)  =  -  [A(t)  -  B(t)F(t)]T  V(t) 


-  V(t)  [A(t)  -  B(t)F(t)  ] 


£(t)  -    FT(t)R(t)F(t)  (A. 13) 


(15)  is  obtained  by  substituting  F(t)  =  P(t)C(t)  in  (A. 13); 
and  (16)  is  obtained  by  letting  t  =  tf  in  (A. 10).   Taking 
the  transpose  of  both  sides  of  (A. 10)  does  not  change  the 
right  side,  hence  V(t)  is  symmetric.   That  V(t  )  is  positive 
definite  can  be  seen  by  substituting  (14)  and  (8)  in  (13) 
and  noting  that  K(t  )  is  positive  definite. 
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